The performance of a Bayesian value-based sequential clinical trial design in the presence of an equivocal cost-effectiveness signal: evidence from the HERO trial

Background There is increasing interest in the capacity of adaptive designs to improve the efficiency of clinical trials. However, relatively little work has investigated how economic considerations – including the costs of the trial – might inform the design and conduct of adaptive clinical trials. Methods We apply a recently published Bayesian model of a value-based sequential clinical trial to data from the ‘Hydroxychloroquine Effectiveness in Reducing symptoms of hand Osteoarthritis’ (HERO) trial. Using parameters estimated from the trial data, including the cost of running the trial, and using multiple imputation to estimate the accumulating cost-effectiveness signal in the presence of missing data, we assess when the trial would have stopped had the value-based model been used. We used re-sampling methods to compare the design’s operating characteristics with those of a conventional fixed length design. Results In contrast to the findings of the only other published retrospective application of this model, the equivocal nature of the cost-effectiveness signal from the HERO trial means that the design would have stopped the trial close to, or at, its maximum planned sample size, with limited additional value delivered via savings in research expenditure. Conclusion Evidence from the two retrospective applications of this design suggests that, when the cost-effectiveness signal in a clinical trial is unambiguous, the Bayesian value-adaptive design can stop the trial before it reaches its maximum sample size, potentially saving research costs when compared with the alternative fixed sample size design. However, when the cost-effectiveness signal is equivocal, the design is expected to run to, or close to, the maximum sample size and deliver limited savings in research costs. Supplementary Information The online version contains supplementary material available at 10.1186/s12874-024-02248-9.

2. Randomly sort this resample and place it into 25 blocks of 10 pairwise allocations (24 blocks of 10 pairwise allocations, plus one block of 8 pairwise allocations).
3. For i = 2, . . ., 25 blocks, impute missing values as follows: (a) use all patients in blocks 1, . . ., i (e.g. for block 8, use the 80 hydroxychloroquine patients and the 80 placebo patients in blocks 1-8); (b) if all patients have a value of either 0 or missing for a given variable, then replace the missing values with 0 (this step is required primarily for some of the cost variables in the first few blocks); (c) identify variables with missing values that will lead to severe collinearity in the univariate models fitted as part of the multiple imputation by chained equations (MICE) procedure [1,2] and exclude them from that procedure (this generally applied to cost variables in early blocks, where the majority of values are either 0 or missing, with only a small proportion of non-zero, non-missing values); (d) impute incomplete variables using MICE [1,2], generating five datasets where these variables are complete.The variables included in the imputation model are described in Appendix Table 1; (e) impute missing values in variables excluded from the MICE procedure (if any) using hotdeck multiple imputation (i.e.missing values are imputed with observed values in proportions that match the observed data, and imputed values can vary within individual between imputed datasets).
4. For blocks 2,. . .,25, calculate a point estimate of expected value of incremental net monetary benefit for each of the five imputed datasets and combine these point estimates by 1 taking their mean (as per Rubin's rules [3,4,5]), resulting in point estimates of expected value of incremental net monetary benefit for blocks 2, . . .,25.
5. Use the point estimates of the expected value of incremental net monetary benefit for each block and the block sizes, together with Bayesian updating (normal prior and likelihood and known variance) to calculate a path of the prior/posterior mean of expected value of incremental net monetary benefit.

Appendix B Choices of parameter values
The parameter values used for the analysis and presented in Table 1 of the main manuscript are sourced and calculated as follows: 1. Estimate of fixed costs of adopting hydroxychloroquine: these were estimated to be zero.
2. Estimate of the sampling standard deviation, σ X .We considered two possible values.
The first is based on the complete cases and is equal to £7632.The second obtains an estimate from the multiple imputation analysis by using Rubin's rules and is equal to £7615.In view of the facts that: 1. the two values are so similar and 2. there is a large amount of missing data in the study, we used the latter value.
3. Estimate of P , the number of patients affected by the adoption decision.This requires an estimate of the incidence rate of the condition and the time horizon over which the decision will apply.P may then be estimated according to: where T is the time horizon and δ is the discount rate [7,8,9].
We consulted a range of publications to estimate I t [10,11,12,13].To meet the eligibility criteria for the HERO trial patients needed to be aged 18 or over and have OA of the first carpometacarpal (CMC) joint and symptomatic OA affecting other hand joints [14].Neither [14,15] nor [6] provide an estimate of the rate at which patients in the United Kingdom report with such symptoms.Using the NHS Hospital Episode Statistics database [16], [12]  done.This gives an estimate of I t = 2450.For the time horizon and discount rate, we followed the approach taken by [17] and set T = 10 and δ = 0, so that P = 24500.
4. Estimate of c, the marginal cost per pairwise allocation, is calculated using the financial records from the trial.Approximately £90,216 was spent prior to recruiting the first patients.This is classified as the fixed set-up cost of the trial.An estimated 50% of the £409,161 of costs incurred between the start of recruitment and the end of followup is taken to be the variable cost of the trial, giving an estimate of the marginal cost per pairwise allocation of £204,580/124 = £1,650.The remaining 50% is taken to be a cost (such as overheads) which would have been incurred during the recruitment phase even if no patients were being recruited.Finally, costs of £336,042 were incurred post follow-up.This gives a total spend of £835,419.
5. Estimate of τ , the delay (measured by the number of pairwise allocations) in observing cost-effectiveness at one year.The trial recruited 248 patients (124 patient pairs) over a period of 611 days (between 24 September 2012 and 27 May 2014).So one pair of patients was randomised approximately every 5 days.This equates to a value of τ ≈ 74 pairwise allocations in one year.Comparison of this estimate with the recruitment profile in [17] shows reasonable agreement (the actual number recruited in the first year was approximately 149).

Appendix C Sequential HERO trial cost-effectiveness data
Columns (3), ( 4) and ( 5) of Appendix Table 2 provide point estimates of incremental Costs, QALYs and net monetary benefit at one year as evidence accumulated (arranged into blocks of 10 pairwise allocations unless stated otherwise).These estimates are based on the multiply imputed data and are cumulative (e.g. the estimates reported for block 10 are based on the observed/imputed data from the 100 pairs observed up to and including this block).The figures in column ( 6) are estimates of the expected value of incremental net monetary benefit for each individual block.Column (7) gives the path of the posterior mean of the expected value of incremental net monetary benefit calculated by combining the by block estimates of incremental net monetary benefit given in column (6) (including the prior) using Bayes rule.
( Appendix Table .2:Cost-effectiveness data at one year as evidence accumulated (based on multiply imputed data and assuming a maximum willingness to pay of £30,000 per QALY.)

( 7 )
Number of Average incremental QALY Average incremental cost (£) Average INMB ( Appendix Table.1:Details of variables included in imputation model used for the HERO trial base case analyses and the analyses reported in this paper would have met the study's inclusion criteria, we assumed that half of them would have (* = medication/appointments relating to hand pain/hand osteoarthritis symptoms only)